The Information Content of Turbulence 
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Information theory has been wonderfully successful in enhancing communication systems, but 
is rarely applied directly to natural phenomena. This work is one of the few attempts to treat a 
physical system as an information source that can be controlled experimentally. It may be the first 
to do so explicitly over a wide range of parameters. The system here is the velocity fluctuations in a 
turbulent soap film. The compressed file size of a velocity record is an estimate of the entropy and 
depends on the Reynolds number. The results are compared with a parallel analysis of the logistic 
map. 
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The equations governing turbulent flows are well 
known, and there is no longer much doubt about their 
applicability. However they yield few exact results that 
can be compared with experiment [l[. The main rea- 
son is that a strongly turbulent velocity field implies the 
nonlinear coupling of a very large number of degrees of 
freedom, making exact solutions elusive and their impli- 
cations even more so. 

Rather than comparing the experimental observations 
with predictions based on the Navier- Stokes equations, 
the measurements are cast in the language of informa- 
tion theory. Just as the contents of a book or a photo- 
graph can be viewed as an information source that can be 
coded and stored in a computer, so can a velocity field. 
One may define the information content of such records 
and ask questions about this information as a function of 
turbulence parameters such as the Reynolds number. 

There has been a significant amount of theoretical work 
developingthe framework of communication and infor- 
mation ■ Information theory has also revealed inter- 
esting new ways of thinking about physics ideas ranging 
from Brownian motion to the uncertainty principle Q. 
However, there have been few attempts to consider ex- 
perimental data as a source of information. There have 
been even fewer such studies of turbulent flows 0-0] ■ 

This work is a study of two-dimensional turbulent ve- 
locity fluctuations in a soap film using information the- 
ory. Moreover, unlike previous work, we consider the 
information content of the velocity fluctuations for a 
wide range of Reynolds numbers. The analysis reveals 
a new way of thinking about turbulence as an informa- 
tion source and not as a solution to the Navier-Stoke's 
equations. The results are strikingly difi^erent than those 
for a parallel analysis of the logistic map. 

Two questions about stored velocity information are 
addressed here: (a) what is the the compressibility of 
this information, that is, how much can its space in the 
computer's memory be reduced, and (b) how often do 
different velocity fluctuation patterns appear? A simi- 
lar issue concerns the frequency of various words in texts 
written in typical books or newspapers. For texts written 



in English (or any language) one tabulates the frequency 
of words and then ranks the probability of their occur- 
rence p{k) in descending order. To illustrate, "the" will 
have rank fc = 1 and "say" will have k = 34. This ranking 
p{k) has been carried out for very many English words. 

Starting with the work of Zipf Q , much attention has 
been devoted to the functional form of p{k). The striking 
observation of many such ranked events (texts in various 
languages, city sizes, etc.) is their power law form: 



p{k) oc k 



(1) 



with a very close to unity for several decades in k. The 
exponent then increases as it must for the sum of p(fc) to 
converge to unity. 

Here we report similar measurements where the 
"words" are groups of velocity fluctuations of various 
magnitudes measured sequentially in time taken from a 
continuous velocity record u(i). For example, the group 
{...Ui, Ui+i, Mi+2,...) = (...-34.52, -10.73, +5.49,...) cm/s 
is here regarded as a word with three letters or symbols 
(L = 3). This permits measurement of the dependence of 
a on L and turbulence parameters such as the Reynolds 
number Re. It is the initial value of the exponent oipik) 
that will be referred to as a. 

Before addressing the information compressibility 
question, one must deflne the information content of a 
message, be it words, a photograph, sentences or veloc- 
ity fluctuations. All of these can be coded, stored, and 
transmitted. We deflne a message as a large collection of 
symbols. A measure of the information or "surprise" con- 
tained in a symbol is log(l/p), where p is the probability 
of that symbol. The base of the logarithm depends on the 
number of different symbols in use and is conventionally 2 
(for binary) . Using the logarithm allows the deflnition of 
information to be additive when combining (multiplying) 
probabilities. If a symbol is very hkely, such as a sunny 
weather forecast in the summer, then the surprise is small 
and the amount of information gained by receiving the 
symbol is also small. The quantity log(l/p) is thus small 
for high probability symbols. The average information 
of the entire message is the information entropy, usually 
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called H 

i7=(logJ-)=^pWlog^, (2) 

where p{i) is the probability that the symbol i is en- 
countered in a long string of symbols. This treatment of 
messages and many useful theorems relating to it is the 
work of C.E. Shannon 0. 

Shannon's work forms the basis of all compression 
schemes used in communication theory, including the 
storage programs utilized by all computers. The com- 
pressibility of a message, according to the Shannon the- 
ory, depends on the probability distribution of the sym- 
bols used. A long string of Is or Os can be compressed 
to almost zero, as can a perfectly repetitious string of Os 
and Is (...101010...). The entropy here is zero, since H 
= -llog(l) = 0. In the opposite limit, a string of Is and 
Os, each chosen randomly by the flip of an unbiased coin, 
cannot be compressed at all and the entropy is a max- 
imum. Shannon recognized that the proper measure of 
the degree of compressibility of a message is its entropy. 
Practically speaking, however, the entropy is the lower 
bound on the compressed length of a message [1, Q . 

Shannon's theorem is valid for any coding method, but 
the calculated entropy and compressibility will depend on 
what coding is used. The true entropy must be calculated 
in such a way that all interdependencies of a message are 
included. Any correlation will reduce the information 
content of a message since knowing one symbol and its 
correlations can spoil the surprise of the next symbol. For 
example, it is very likely that "ver" will be followed by 
"y" in an English text. Likewise, using letters as the base 
set of symbols for coding a book yields an overestimate 
of the entropy since this overlooks word and sentence 
structures. When we only know our letters and are first 
beginning to read, any book looks random! Using words 
and correlations between words for coding will decrease 
the calculated entropy and increase the compressibility. 

There are many algorithms that achieve close to op- 
timal compression and so will approach the best es- 
timate of the entropy currently possible using such a 
method 0, [l3| . They of necessity yield message lengths 
slightly larger than H. For the data in the logistic map 
and for the laboratory-acquired data, the widely invoked 
Lempel-Ziv compression method is used in the form of 
the .zip compression algorithm We emphasize that 
calculating — '^^p{i) \ogp{i), where p{i) is the probabil- 
ity of a single velocity value, has been done countless 
times before but is not a good estimate of the entropy. 

To clarify the methods of analysis to be discussed, it is 
instructive to apply the same tools to a simple nonlinear 
map, namely the logistic map defined as 

Xn+l = r[Xnil - Xn)], (3) 

where r can span the interval and 4. The initially 



chosen value xq is taken to lie in the interval < .tq < 1 , 
as do all Xn- 

Since the logistic map is well known jTll | , it will not be 
discussed here other than to say that for r larger than 
a certain critical value, all the Xn take on many values, 
jumping from one Xn to the next in a seemingly random 
way, though of course Eq. ^ is deterministic. One is 
tempted to think of r in this equation as analogous to 
the the Reynolds number Re with r = 4 corresponding 
to Re — >■ oo. As will be seen however. Re and r behave 
quite differently as control variables. 

For values of r less than 3.57, almost any choice of 
Xq will lead to a fixed value of Xn or produce periodic 
oscillations. So for this regime the logistic map is almost 
infinitely compressible, that is, ~ and the rank p{k) 
has no meaning. The focus here will be on slightly larger 
values than this. 

Equation ^ is invoked for 10^ values of n starting with 
randomly chosen seed values xq- The data are analyzed 
by subtracting the average value (a;„) ~ 0.5 from each 
entry x„ and dividing by the rms value Xrms for all 10^ 
data points. Each number is then rounded to several 
significant figures (in decimal units) and a short string of 
these data values are treated as a word. This method is 
chosen for continuity with the analysis of the turbulence 
measurements discussed below. 

One finds that the Zipf exponent a for the logistic map 
is a slowly increasing function of the rank k, as in English 
texts. All curves have a similar slope, a = 0.2 for the first 
decade in k (data not shown). This slope depends on r. 
For r = 3.59, a = 0.5 and for r = 3.61, a = 0.25. It is 
notable that a is much less than unity, contrary to its 
value for the ranking of words in various languages, for 
city population sizes, earthquake magnitudes, and many 
other tabulations of activities created by nature and by 
human activity jl2|. 

Figure [T] is a plot of the compression ratio C(r) vs. r, 
where C(r) is defined as the ratio of the file size of the 
compressed record to its uncompressed size. This varies 
between and 1 and is bounded below by H (normal- 
ized). The message under compression is the 10^ values 
of (normalized) Xn obtained at closely-spaced values of r 
in the range 3.3 < r < 4. 

For r below 3.4, C(r) is near 0, as expected. This is 
followed by a small increase above zero after period dou- 
bling commences at r ~ 3.45. Except for a few values of 
r, were C(r) < 0.02, the fractional amount of compres- 
sion increases linearly with r in the narrow interval 3.58 
< r < 3.7 (solid straight line in this linear plot). 

Above r ~ 3.7, C(r) is rather constant until r reaches 
its maximum value of 4. In this interval C(r) fluctuates 
about a mean value of approximately 0.17. There are 
several points in this range of r for which C(r) is very 
small. These are the well known "islands of stability" 
that occur for the logistic map in the normally chaotic 
regime, where for certain values of r the system returns 
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FIG. 1: Compression ratio C{r) versus the logistic map pa- 
rameter r. After the transition to chaos, the compression (and 
entropy) increases. The solid line has a slope of roughly 1.1. 



to periodicity 

Apparently, the chaotic regime of the logistic map 
is becoming increasingly random, since the entropy in- 
creases the more random a signal becomes. However, we 
do not understand the linear variation with r during the 
transition interval. The turbulence measurements will 
show very different behavior. 

We now consider the same analysis applied to the tur- 
bulent velocity measurements in our 2D soap film flow. 
A soap solution is fed to the film from a reservoir and is 
driven by gravity. The film is drawn between two thin 
metal blades separated by a distance w of the order of 
several cm and the turbulence is generated by a comb 
inserted horizontally into the flow. The mean flow speed 
U is of the order of 100 cm/s. More experimental details 
can be found in previous work using this system [13 1. 

Very long velocity time series (> 10^) of the stream- 
wise velocity component are taken with an LDV (laser 
Doppler vclocimeter) at the lateral center of the soap film 
channel and interpolated for equal time spacing. The av- 
erage data rate is between 1 kHz and 10 kHz. 

The velocity data are then treated in the same way as 
the logistic map data with the mean subtracted out and 
the resulting difference divided by the rms value. The 
velocity record is then saved to several significant figures 
in decimal units. It is important not to have too few 
significant figures nor too many. Too few will lead to 
a coarse-grained description that throws out important 
details just like a grainy photograph. In the opposite 
limit if there are too many, statistical fluctuations are 
increased due to the limited length of the entire data 
string. 

Figure [5] is a linear-log plot of C versus a Reynolds 
number Re' = u'w/v where w is the channel width, u' is 
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FIG. 2: Compression ratio C versus Reynolds number Re' for 
the 2D turbulent velocity data. The straight line is a loga- 
rithmic fit to the experimental data. The solid line is given by 
the equation C{Re) = -0.003 log(_Re) + 0.34. The data here 
were saved to three significant figures before compression. 



the rms velocity (of order 10 cm/s) and i/ is the kinematic 
viscosity (0.01 cm^/s). If the independent variable, is 
taken as Re = Uw/v rather than i?e', the same behavior 
is seen but the scatter of the data points is increased. 

The compression ratio (and hence H) is clearly of loga- 
rithmic form: C{Re) oc — log(i?e). Note that C{Re) is a 
decreasing function of the Reynolds number, in stark con- 
trast to the logistic map (see Fig. [Ij, where it increases 
with r after turning chaotic. This is also contrary to the 
results of Wijesekera Q , who estimated H via the power 
spectra of density fluctuations in the ocean. However, it 
is difficult to compare given their unique definition of the 
Re. 

This decrease in C{Re) suggests an increase in the 
number of correlated spatial and temporal scales as Re 
grows There is reason to expect the opposite be- 
havior, since the number of deg rees of freedom should 
increase with increasing Re [1J|; all else being equal a 
larger alphabet of symbols means a larger entropy. The 
decrease in C{Re) is occurring in spite of the growing 
number of degrees of freedom that are excited as the flow 
becomes more turbulent. 

In the higher Reynolds number range {Re > 10^) the 
measured power spectrum of the velocity fluctuations 
E(k) {k is the wavenumber) has a clear power law scal- 
ing with an exponent near -3 (spectral data not shown). 
This value is indicative of the direct enstrophy cascade 
usually observed in decaying 2D turbulence [l3|. The 
power spectra measured for Re < 10^ are flat, indicating 
that there is no self-similar scaling. It is striking that 
the present method of data analysis is able to capture 
the behavior of turbulence at Re too low for the usual 
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Kolmogorov scaling ideas to be applied [l[ . 

The analysis of the probability-rank distributions can 
also be applied to the measurements. The distributions 
are plotted in Figure [3] for widely different values of Re 
using a word length of L = 3. A somewhat longer word 
length could have been used. To the extent that the data 
can be characterized by a power law exponent, it is much 
less than the Zipf value a ~ 1. 
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FIG. 3: Word probability p{k) versus rank k for several differ- 
ent Reynolds numbers, where Re = Uw/v. The word length 
is L = 3. The broadness of the distribution decreases with in- 
creasing Reynolds number. (□) Re = 49516, (O) Re = 13172, 
(A) Re = 1823. The dashed hne is only a guide for the eye. 

Since the entropy is calculated from a probability dis- 
tribution, the functional form of C{Re) is presumably 
connected with p{k). Of course, these p{k) are not the 
distributions for the coding that the compression algo- 
rithm actually uses. Note that the broadness of the 
distributions in Figure |3] decreases with increasing Re. 
This is consistent with the compression data; increasing 
Re increases constraints (correlations) which decreases 
C{Re) and should narrow p{k). Stated in another way, 
the broadness of a distribution, such as is linked 

to the magnitude of the entropy since a completely flat 
distribution gives the maximum entropy. 

In summary, the velocity time series of a 2D turbulent 
soap film is analyzed using tools from Shannon's infor- 
mation theory. The main result is that the compression 
ratio, which is an approximation to the true entropy, is 
measured to be a logarithmically decreasing function of 
the Reynolds number. As far as we know, there are no 
predictions for the dependence of Shannon's entropy on 
the Reynolds number, although the decrease is in accord 
with our expectations. This strikingly weak dependence 
of the compression ratio on the strength of the turbulence 
calls for a simple explanation, but so far it has eluded us. 

These measurements include low Reynolds number 



turbulence where the cascade picture can not be applied. 
If the experiments were expanded to include laminar 
fiows, the compression ratio would start out as almost 
zero. Assuming that the entropy is a continuous func- 
tion, it must then increase with Reynolds number, just 
as the logistic map analyzed here and as in H, Q- It 
should reach a maximum before decreasing logarithmi- 
cally in the manner of Figure [2j This slow decrease sug- 
gests that increasing the Reynolds number further may 
only affect the turbulence by a small amount. This is in 
accord with our present understanding of large Reynolds 
number turbulence. 

Many of the tools developed under the umbrella of in- 
formation theory have not yet been applied to experi- 
mental data. Shannon entropy and other measures of 
complexity [l5| are, for the most part, waiting to be used, 
although there has been some work already done (B-Q]. 
We believe the present experiments are amongst the first 
few steps in that direction. 
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